Presentation on Rambachan and Roth (2022)1

Chencheng Fang

2022-11-07

Limitation of Pre-trend Testing2

  1. 🙅 Low Power: Pre-trend test may fail to detect violations of parallel trends.

  2. 🙅 Distortion: Selection bias from only analyzing cases with insignificant pre-trend.

  3. 🙅 Parametric approaches to controlling for pre-existing trends are Sensitive to functional form.

    💁 What if pre-trend test fails? Could we still make valid inference? 

Intuition

Example: Violation of Parallel Trends

General Idea: Sensitivity of causal conclusions to alternative assumptions on the possible violations of parallel trends

Possible choices of \(\Delta\)

  1. Bounding Relative Magnitudes: \[\Delta^{RM}(\bar{M}):=\{\delta: \forall t \ge 0, |\delta_{t+1}-\delta_t|\le \bar{M}\cdot \max_\limits{s<0} |\delta_{s+1}-\delta_s|\}\]
  2. Smoothness restrictions: \[\Delta^{SD}(M):=\{\delta: |(\delta_{t+1}-\delta_t)-(\delta_{t}-\delta_{t-1})|\le M, \forall t\}\]
  3. Combining the two restrictions above: \[\Delta^{SDRM}(\bar{M}):=\{\delta: \forall t \ge 0, |(\delta_{t+1}-\delta_t)-(\delta_{t}-\delta_{t-1})|\le \bar{M}\cdot \max_\limits{s<0} |(\delta_{s+1}-\delta_s)-(\delta_{s}-\delta_{s-1})|\}\]
  4. Sign restrictions (take positive sign as an example): \[\Delta^{PB}:=\{\delta:\delta_t \ge 0 \quad \forall t\ge0\}\]
  5. Monotonicity restrictions (take increasing monotonicity as an example): \[\Delta^{I}:=\{\delta:\delta_t \ge \delta_{t-1} \quad \forall t\}\]

Wrap-up all possible \(\Delta\): Polyhedral restrictions

👏 All restrictions discussed can be wrapped up in a nutshell of Polyhedral restrictions or a finite unions of such restrictions. \[\Delta=\{\delta: A\delta \le d\}\] for some known matrix \(A\) and vector \(d\).

📚 Example Again: \(\Delta^{SD}(M)\) in three period DiD setting

\[\Delta^{SD}(M):=\{\delta: |(\delta_{1}-\delta_0)-(\delta_{0}-\delta_{-1})|\le M\}\] Equally, \(\Delta^{SD}(M):=\{\delta: A^{SD}\delta \le d^{SD}\}\) for \(A^{SD}=\left( \begin{array}{cc} 1&1 \\ -1&-1 \end{array} \right)\) and \(d^{SD}=(M,M)^{\prime}\)

Inferential Goal: Confidence Set with Uniform Validness

How to Get such a Confidence Set?

1️⃣ Inference using Moment Inequalities

2️⃣ Inference using Fixed Length Confidence Intervals

Inference using Moment Inequalities

Defining null hypothesis and rewriting it as a moment inequality problem

Inference using Moment Inequalities

Inference using Moment Inequalities

Constructing conditional confidence sets

If \(\gamma_{\star}\) is optimal in dual problem, it is a vector of Lagrangian multipliers in primal problem.

where \(\xi\sim \mathcal{N}\big(\gamma^{\prime}_{\star}\tilde{\mu}(\bar{\theta}),\gamma^{\prime}_{\star}\tilde{\Sigma}_n\gamma_{\star}\big)\), \(\tilde{\mu}(\bar{\theta})=\mathbb{E}\big[\tilde{Y}_n(\bar{\theta})\big]\), \(\hat{V}_n \subset V(\Sigma_n)\),\(S_n=(I-\frac{\tilde{\Sigma}_n\gamma_{\star}}{\gamma^{\prime}_{\star}\tilde{\Sigma}_n\gamma_{\star}}\gamma^{\prime}_{\star})\tilde{Y}_n(\bar{\theta})\) and \(v^{lo}\), \(v^{up}\) are known functions of \(\tilde{\Sigma}_n\),\(s\) and \(\gamma_{\star}\).

Inference using Moment Inequalities

Constructing conditional confidence sets

⭕ Case 1: In population, one moment is violated and others are very slack,i.e., \(\tilde{\mu}_1>0\) while \(\tilde{\mu}_j \ll 0\) for \(j \not=1\).

❌ Case 2: \(\mu_1 \approx \mu_2 >0\). Low in power.

Inference using Moment Inequalities

Constructing hybrid confidence sets

1️⃣ A size-\(\kappa\) (Usually \(\kappa=\alpha/10\)) LF test that rejects whenever \(\hat{\eta}\) exceeds the \(1-\kappa\) quatile of \(\max_{\gamma\in V(\Sigma)}\gamma^{\prime}\xi\), where \(\xi\sim \mathcal{N}(0,\tilde{\Sigma}_n)\). The “hybrid” approach rejects if size-\(\kappa\) rejects. Otherwise, proceed to stage 2.

2️⃣ A size-\(\big(\frac{\alpha-\kappa}{1-\kappa}\big)\) conditional test that also conditions on the event that the first stage LF test did not reject. By defining \(v^{up}_H=\min\{v^{up}, c_{LF,\kappa}\}\), we have: \[\hat{\eta}| \{\gamma_{\star} \in \hat{V}_n, S_{n}=s, \hat{\eta}\le c_{LF,\kappa}\}\sim \xi| \xi \in [v^{lo},v^{up}_H]\]

Inference using Moment Inequalities

Asymptotic Properties

Besides finite sample size control, following asymptotic properties of conditional and hybrid approaches have been proven.

Inference using Fixed Length Confidence Intervals

Inference using Fixed Length Confidence Intervals

Constructing FLCIs

\[\hat{\beta}_n \sim \mathcal{N}(\beta,\Sigma_n)\]

\(\Longrightarrow |a+v^{\prime}\hat{\beta}_n-\theta|\sim |\mathcal{N}(b,v^{\prime}\Sigma_nv)|\), where \(b=a+v^{\prime}\hat{\beta_n}-\theta\)$

\(\Longrightarrow \theta \in \mathcal{C}_{\alpha,n}(a,v,\chi) \quad \textrm{iff} \quad |a+v^{\prime}\hat{\beta}_n-\theta|\le\chi\)

\(\Longrightarrow\) For fixed \(a\) and \(v\), the smallest \(\chi\) that satisfies coverage requirement is \(1-\alpha\) quantile of \(|\mathcal{N}(\bar{b},v^{\prime}\Sigma_nv)|\), where \(\bar{b}\) is the worst case bias.

\(\Longrightarrow\) \(\chi_n(a,v,\alpha)=\sigma_{v,n}\cdot cv_{\alpha}(\bar{b}(a,v)/\sigma_{v,n})\), where \(cv_{\alpha}(t)\) is \(1-\alpha\) quantile of \(|\mathcal{N}(t,1)|\) and \(\sigma_{v,n}:=\sqrt{v^{\prime}\Sigma_nv}\).

\(\Longrightarrow\) \(\mathcal{C}_{\alpha,n}^{FLCI}(\hat{\beta}_n,\Sigma_n):=(a_n+v_n^{\prime}\hat{\beta}_n)\pm\chi_n\), where \(\chi_n:=\inf_{a,v}\chi_n(a,v,\alpha)\)

Inference using Fixed Length Confidence Intervals

Finite Sample near Optimality

(In)Consistency of FLCIs

Application

Sensitivity Analysis

Simulation

Finite-sample Properties

Simulation

Finite-sample Properties

Simulation

Finite-sample Properties

Additional Resources

Thank you!

Contact:

Twitter:

GitHub:


  1. Rambachan, A. and Roth, J. (2022), A more credible approach to parallel trends, Forthcoming, Review of Economic Studies↩︎

  2. Roth, J.(2022), Pretest with Caution: Event-Study Estimates after Testing for Parallel Trends, American Economic Review: Insights, 4 (3), 305–322↩︎

  3. Andrews, Isaiah, Jonathan Roth, and Ariel Pakes (2022), Inference for Linear Conditional Moment Inequalities, Forthcoming, Review of Economic Studies↩︎

  4. Bailey, M.J. and Goodman-Bacon, A. (2015),The War on Poverty’s Experiment in Public Medicine: Community Health Centers and the Mortality of Older Americans, American Economic Review, 105 (3), 1067–1104↩︎